Semiconductor based storage has traditionally been employed as the main storage component of computers, and as the storage medium ill cached disk control units. In On-Line Transaction Processing (OLTP) systems much of the main storage is used to buffer disk storage data blocks, an application that is similar in character to the cacheing functions of disk control units. Buffering and cacheing minimize the numbers of physical disk accesses by intercepting READ requests to disk, providing the requested data from main storage buffers or from caches in the disk controller. Main memory buffering can substantially reduce the burden on the computer's I/O subsystem and on the disk actuators since READ requests that are intercepted by main memory buffering result in no I/O or disk activity. READ requests which are satisfied from disk controller caches still require the computer to initiate I/O channel operations but they eliminate actual disk actuator activity, and since the latency for a READ that is satisfied from cache is small, I/O channel occupancy is greatly reduced. Reduction in physical disk activity is particularly important, since each disk actuator can only service from 20 to 40 random access requests per second, the exact number depending upon the disk model and specific access patterns. As processor speeds improve and transaction rates and complexity increase, it thus becomes important to minimize the number of physical disk accesses so that they can be satisfied by an economic number of disk actuators.
For example, if a system is processing 1000 transactions per second and each transaction accesses (READS or WRITES) 40 data items, then without cacheing or buffering the disk subsystem would need to support 40,000 disk operations per second. If these accesses could be spread without skew across all available disk actuators, then some 2000 disk actuators would be required. Skew effects make this a minimum requirement ill most systems. If 90% of the READ requests can be intercepted by main memory buffers or disk caches, and if only 15% of the access requests are WRITES, then the burden on disk actuators is reduced by 75%. Note that both techniques, in addition to reducing disk accesses, substantially reduce the latency to availability of the requested data. This results in greater efficiencies and less potential conflict between parallel execution of transactions, and the multi-programming levels within the computer can be correspondingly reduced. It also reduces transaction response times. Cacheing within the disk control unit, which has somewhat greater latencies than main memory buffering, has a significant advantage over main memory buffering when the disk data is being shared among several computers, since it permits straightforward sharing of data. With main memory buffering, this is not straight forward as some mechanism must be provided for invalidating buffered data in one computer when the underlying disk data is modified by another computer.
It should be noted that as buffer and cache sizes increase, a larger fraction of READ requests are intercepted. Ultimately with large enough memories, virtually all READ requests can be satisfied from buffers or cache. In such systems, disk activity is dominated by the need to WRITE all updates, modifications, or additions through to disk. This requirement to reflect all writes to disk is driven by the vastly better integrity characteristics of disk (magnetic) storage when compared to conventionally designed main storage or disk controller cache memory (semiconductor storage). Since write or update activities constitute a significant fraction of disk requests for most OLTP workloads, the sizing and throughput of a conventional OLTP system is thus ultimately limited by the capacity and characteristics of the supporting disk actuators, even when vast amounts of semiconductor memory are installed.
A primary motivation for this invention is thus to improve the integrity characteristics of a large semiconductor based storage subsystem so that it can be used for storage of data without reflecting modifications or updates to disk. Since disk data is frequently mirrored (duplexed) in contemporary systems, such a memory must be fault-tolerant and non-volatile to compete effectively against disk based storage's integrity profile. This permits a database to entirely reside in this semiconductor memory, without any disk backing. It also allows write-back cacheing of disk based databases. Write-back cacheing, instead of write-through cacheing provides continued reductions in the numbers of disk accesses with the addition of memory since both READS and WRITES can be intercepted. Ultimately all disk accesses are effectively eliminated when the cache becomes large enough.
The preferred embodiment of this invention, disclosed herein, additionally seeks to position this fault-tolerant and non-volatile memory so that it can be easily shared among a plurality of client computers, facilitating the construction of very large transaction processing systems. Toward this goal, fault-tolerant, processor components are embedded within the fault-tolerant non-volatile memory to provide an intelligent fault-tolerant non-volatile memory subsystem that can be shared among several client computers for shared cacheing of data, and for the complete storage of data bases that will fit.